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A search for the lepton-flavor-violating decay of the tau into three charged Icptons has been 
performed using 91.5 fb~^ of data collected at an e^e~ center-of-mass energy of 10.58 GeV with 
the BABAR detector at the PEP-II storage ring. In all six decay modes considered, the numbers 
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of events found in data are compatible with the background expectations. Upper limits on the 
branching fractions are set in the range (1 - 3) X 10"^ at 90% confidence level. 



PACS numbers: 13.35.Dx, 14.60.Fg, ll.30.Hv 

Lepton-flavor violation (LFV) involving charged lep- 
tons has never been observed, and stringent experimental 
limits exist from muon branching fractions: B{ii — *■ ej) < 
1.2 X 10-" H] and ^ eee) < 1.0 x IQ-^^ Q] at 90% 
confidence level (CL). Recent results from neutrino oscil- 
lation experiments 'J\ show that LFV does indeed occur, 
although the branching fractions expected in charged lep- 
ton decay due to neutrino mixing alone are probably no 
more than lO"^'' 0. 

In tau decays, the most stringent limit on LFV is 
B{t ^ ^7) < 3.1 X lO-'^ at 90% CL Q. Many extensions 
to the Standard Model (SM), particularly models seeking 
to describe neutrino mixing, predict enhanced LFV in tau 
decays over muon decays with branching fractions from 
10"^'^ up to the current experimental limits 0. Obser- 
vation of LFV in tau decays would be a clear signature 
of non-SM physics, while improved limits will provide 
further constraints on theoretical models. 

This analysis is based on data recorded by the BABAR 
detector at the PEP-II asymmetric-energy e+e~ storage 
ring operated at the Stanford Linear Accelerator Center. 
The data sample consists of 81.9 fb^^ recorded at ^/s — 
10.58 GeV and 9.6 fb~^ recorded at Vs = 10.54 GeV. 
With an expected cross section for tau pairs at the 
luminosity- weig hted Vs of arr = (0.89 ± 0.02) nb 0, 
this data sample contains over 160 million tau decays. 

The BABAR detector is described in detail in Ref. 
Charged-particle (track) momenta are measured with a 
5-layer double-sided silicon vertex tracker and a 40-layer 
drift chamber inside a 1.5-T superconducting solenoidal 
magnet. The transverse momentum resolution param- 
eterized as aprp/pT = (0.13 • pT/[GeY/c] + 0.45)% is 
achieved. An electromagnetic calorimeter consisting of 
6580 CsI(Tl) crystals is used to identify electrons and 
photons, a ring-imaging Cherenkov detector is used to 
identify charged hadrons, and the instrumented magnetic 
flux return (IFR) is used to identify muons. Particle at- 
tributes are reconstructed in the laboratory frame and 
then boosted to the e+e" center-of-mass (CM) frame us- 
ing the measured asymmetric beam energies. 

This paper presents a search for LFV in the neutrino- 
less decay r~ — > £~£~^£~. All possible lepton combina- 
tions consistent with charge conservation are considered, 
leading to six distinct decay modes (e^e+e^, /i+e^e^, 
/j,~e"'"e", e'^fi^fi^ , e~^~^^~ , |9|. The signature 

of this process is three charged particles, each identi- 
fied as either an electron or muon, with an invariant 
mass and energy equal to that of the parent tau lep- 
ton. Candidate signal events in this analysis are required 
to have a "1-3 topology," where one tau decay yields 
three charged particles (3-prong), while the second tau 



decay yields one charged particle (1-prong). Four well 
reconstructed tracks are required with zero net charge, 
pointing towards a common region consistent with r"'"r~ 
production and decay. One of these tracks must be sep- 
arated from the other three by at least 90° in the CM 
frame. The plane perpendicular to this isolated track di- 
vides the event into two hemispheres and defines the 1-3 
topology. Pairs of oppositely charged tracks identified 
as photon conversions in the detector material with an 
e^e~ invariant mass below 30MeV/c^ are ignored. 

Each of the charged particles found in the 3-prong 
hemisphere must be identified as either an electron or 
muon candidate. Electrons are identified using the ra- 
tio of calorimeter energy to track momentum (E/p), the 
ionization loss in the tracking system (dE/dx), and the 
shape of the shower in the calorimeter. Muons are iden- 
tified by hits in the IFR and small energy deposits in the 
calorimeter. Muons with momentum less than 0.5GeV/c 
cannot be identified because they do not penetrate far 
enough into the IFR. 

The particle identification (PID) requirements are not 
sufhcient to suppress certain backgrounds, particularly 
those from higher-order radiative Bhabha and 
events that can have four leptons in the final state. To re- 
duce these backgrounds, additional selection criteria are 
applied to the six different decay modes. For all de- 
cay modes, the momentum of the 1-prong track is re- 
quired to be less than 4.8 GeV/c in the CM frame. For 
the e~e^e^ and e^/i^/^^ decay modes, the charged par- 
ticle in the 1-prong hemisphere must not be identified 
as an electron, while for the /i~e"'"e~ and /Lt~/x"'"/z~ de- 
cay modes it must not be a muon. For all four of these 
decay modes, the angle Ois between the 1-prong momen- 
tum and the vector sum of the 3-prong momenta in the 
CM frame must satisfy cos 6*13 > —0.9999, while the net 
transverse momentum of the four tracks must be greater 
than 0.1 GeV/c. Additional requirements are imposed to 
reduce the qq and SM r+r~ backgrounds. Events in the 
four decay modes specified above are required to have 
no unassociated calorimeter clusters (photons) in the 3- 
prong hemisphere with energy greater than 100 MeV in 
the laboratory frame, while events in all six decay modes 
are required to have no track in the 3-prong hemisphere 
that is also consistent with being a kaon. 

To reduce backgrounds further, signal events are re- 
quired to have an invariant mass and total energy in the 
3-prong hemisphere consistent with a parent tau lepton. 
These quantities are calculated from the observed track 
momenta assuming the corresponding lepton masses for 
each decay mode. The energy difference is defined as 
AE = E*^^ — i?bcam' where E*^^ is the total energy of the 
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tracks observed in the 3-prong hemisphere and -Bbeam 
the beam energy, both in the CM frame. The mass dif- 
ference is defined as /S.M = il/rcc ^ where M,.oc is 
the reconstructed invariant mass of the three tracks and 
nir = 1.777 GeV/c^ is the tau mass 

The signal distributions in the (AA/, /S.E) plane are 
broadened by detector resolution and radiative effects. 
The radiation of photons from the incoming e^e~ parti- 
cles before annihilation affects all decay modes, leading 
to a tail at low values of AE'. Radiation from the final- 
state leptons is more likely for electrons than muons, and 
produces a tail at low values of AM as well. Rectangular 
signal regions are defined separately for each decay mode 
as follows. For all six decay modes, the upper right corner 
of the signal region is fixed at (30 MeV/c^, 50 MeV), while 
the lower left corner is at (—70,-120) for the e~e+e~ 
and fi~e~^e~ decay modes, (-100,-200) for ^+e~e~, 
(-50,-200) for e+fi-fi-, (-50,-150) for e>+^-, and 
(-30,-150) for fi~fi^fi~ . All values are given in units 
of (MeV/c^, MeV). These signal region boundaries are 
chosen to provide the smallest expected upper limits on 
the branching fractions in the background-only hypothe- 
sis. These expected upper limits are estimated using only 
Monte Carlo (MC) simulations and data control samples, 
not candidate signal events. Figure ^ shows the observed 
data in the (AM, AE) plane, along with the signal re- 
gion boundaries and the expected signal distributions. 
To avoid bias, a blinded analysis procedure was adopted 
with the number of data events in the signal region re- 
maining unknown until the selection criteria were final- 
ized and all cross checks were performed. 

The efficiency of the selection for signal events is esti- 
mated with a MC simulation of LFV tau decays. Sim- 
ulated tau-pair events including higher-order radiative 
corrections are generated using KK2f (31 with one tau de- 
caying to three leptons with a 3-body phase space distri- 
bution, while the other tau decays according to measured 
rates [llj simulated with Tauola Final state radia- 
tive effects are simulated for all decays using Photos [l3j . 
The detector response is simulated with GEANT4 , and 
the simulated events are then reconstructed in the same 
manner as data. 

About 50% of the MC signal events pass the 1-3 topol- 
ogy requirement. The lepton identification efficiencies 
and misidentification probabilities are measured using 
tracks in kinematically-selected data samples (radiative 
Bhabha, radiative jX^ fjL~ , two-photon e^e~i^i~, and 
J I'll) — > t^t'') and parameterized as a function of particle 
momentum, polar angle, and azimuthal angle in the lab- 
oratory frame. These data-derived efficiencies are then 
used to give the probability that a simulated MC parti- 
cle will be identified (or misidentified) as an electron or 
a muon. For the lepton momentum spectrum predicted 
by the signal MC, the electron and muon identification 
requirements are found to have an average efficiency per 
lepton of 91% and 63%, respectively. The probability 
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FIG. 1: Observed data shown as dots in the (AM, AE) plane 
and the boundaries of the signal region for each decay mode. 
The dark and light shading indicates contours containing 50% 
and 90% of the selected MC signal events, respectively. The 
regions shown in Fig. |5] are indicated by dashed lines. 



for a hadron to be misidentified as an electron in SM 
3-prong tau decays is 2.2%, while the probability to be 
misidentified as a muon is 4.8% The final efficiency 
for signal events to be found in the signal region is shown 
in Table ^ for each decay mode and ranges from 7% to 
12%. This efficiency includes the 85% branching fraction 
for 1-prong tau decays. 

There are three main classes of background remaining 
after the selection criteria are applied: low multiplicity qq 
events (mainly continuum light-quark production) , QED 
events (Bhabha and ^^/i^), and SM t~^t~ events. These 
three background classes have distinctive distributions in 
the (AM, AE) plane: qq events tend to populate the 
plane uniformly, while QED backgrounds are restricted 
to a narrow band at positive values of AE, and t+t^ 
backgrounds are restricted to negative values of both AE 
and AM. A negligible two-photon background remains. 

The expected background rates for each decay mode 
are determined by fitting a set of probability density 
functions (PDFs) to the observed data in the (AM, AE) 
plane in a grand sideband (GS) region. The GS re- 
gion, shown in Fig. ^ is defined as the rectangle 
bounded by the points (-600 MeV/c^, -700 MeV) and 
(400MeV/c2,400MeV), excluding the signal region. For 
both the qq and r+r" backgrounds, an analytic PDF is 
constructed from the product of two PDFs and Pe, 
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TABLE I: Efficiency estimates, number of expected back- 
ground events (A'^bgd), number of observed events (A^obs), and 
branctiing fraction upper iimits for eacfi decay mode. 



D6C£iy hiocIb 








KffiripTirv \%] 

ijli II .iv^i II. y / yj \ 


7.3 ± 0.2 


11.6 ± 0.4 


7.7 ± 0.3 


qq ogu. 


U.U i 


fl 1 7 


n 'HQ 


OPn ho-H 

Vc^-Cj-L-* UgCl. 






u. 


t~'"t~ bfd 


0.00 


0.01 


0.00 


Vbgd 


1 ^1 + n 1 1 


u.o ^ m u.uo 


n R9 + n 1 n 

u.uz HZ u. lu 


Nohs 


1 





1 




2.0 X 10"' 


1.1 X lO"' 


2.7 X 10" ' 


Decay mode 








Efficiency [%] 


9.8 ±0.5 


6.8 ±0.4 


6.7 ±0.5 


qq bgd. 


0.20 


0.19 


0.29 


QED bgd. 


0.00 


0.19 


0.01 


t'^t~ bgd. 


0.01 


0.01 


0.01 


Abgd 


0.21 ±0.07 


0.39 ±0.08 


0.31 ±0.09 


iVobs 





1 







1.3 X lO^' 


3.3 X lO^' 


1.9 X 10"' 



S4 



-0.6 



T - 


e"eV 




x" — > |iVe' 

fl fl .fl fl . 


.^^11 . fl . 


I I i . 


III 


t — > e*(i ^ 
_1 i 


■'Jfl . , . 


X - 


e 

fl n. . 


ilmfl n 1 i i .fl 



-0.4 



-0.2 



-0 0.2 2 

A M (GeV/c ) 



-0.4 



-0.2 0.2 ,0.4 

A M (GeV/c") 



FIG. 2: Distribution of AAf for data (solid histogram) and 
background PDFs (solid curves) for events with /S.E in the 
signal region indicated in Fig. Expected signal distribu- 
tions are shown (dashed histogram) for a branching fraction 
of 10"'' . 



where Pm (AM) is the sum of two Gaussians with a com- 
mon mean and Pe{AE) = (l — x/y/l + x'^)(l + ax + bx'^ + 
cx^) with X = {AE-d)/e O- The shapes of these PDFs 
are described by a total of nine free parameters, which 
are determined by fits to MC qq and t~^t~ background 
samples for each decay mode. 

For the QED backgrounds, an analytic PDF is con- 
structed from the product of a Crystal Ball function 
[r^ in AE' and a linear function in AM', where 
the (AM', AE') axes have been rotated slightly from 
(AM, AE) to fit the observed distribution. The six pa- 
rameters of this PDF, including the rotation angle, are 
obtained by fitting control samples with a 1-3 topology 
that are enhanced in Bhabha or jX^ ii~ events by requiring 
that the particle in the 1-prong hemisphere is identified 
as an electron or muon. Any value for cos^ia is allowed, 
but the control sample events otherwise pass the selection 
criteria. 

With the shapes of the three background PDFs deter- 
mined, an unbinned maximum likelihood fit to the data 
in the GS region is used to find the expected rate of each 
background type in the signal region, as shown in Tabled 
The PDF shape determinations and background fits are 
performed separately for each of the six decay modes. 
Figure El shows the data and the background PDFs for 
values of AE in the signal range. 

The largest systematic uncertainty in the signal effi- 
ciency is due to the uncertainty in measuring the PID 
efficiencies. This uncertainty is determined from the sta- 
tistical precision of the PID control samples, and ranges 
from 0.7% for e~e+e~ to 6.2% for /z~^+/.t~ relative to the 
efficiency (l^J. The modeling of the tracking efficiency 
contributes an additional 2% uncertainty, as does the 
statistical limitation of the MC signal sample. All other 
sources of uncertainty are found to be small, including 



the modeling in the generator of radiative effects, track 
momentum resolution, trigger performance, observables 
used in the selection criteria, and knowledge of the tau 
1-prong branching fractions. The efficiency has been es- 
timated using a 3-body phase space model and no un- 
certainty is assigned for possible model dependence. The 
selection efficiency is found to be uniform within 10% 
across the Dalitz plane, provided the invariant mass for 
any pair of leptons is less than 1.4GeV/c^. 

Since the background levels are extracted directly from 
the data, systematic uncertainties on the background es- 
timation are directly related to the background param- 
eterization and the fit technique used. The finite data 
available in the GS region to determine the background 
rates is the largest uncertainty and varies from 10% to 
25% depending upon the decay mode. Additional uncer- 
tainties are estimated by varying the fit procedure and 
changing the functional form of the background PDFs. 
Cross checks of the background estimation were per- 
formed by considering the number of events expected 
and observed in sideband regions immediately neighbor- 
ing the signal region for each decay mode. 

The numbers of events observed (A^obs) and the back- 
ground expectations (A^bgd) are shown in TableQ] with no 
significant excess found in any decay mode. Upper lim- 
its on the branching fractions are calculated according to 
B^j^ = Nl\^/{2eh(Trr), where N%\^ is the 90% CL upper 
limit for the number of signal events when A'obs events 
are observed with A'bgd background events expected. The 
values e, L, and u-tt are the selection efficiency, luminos- 
ity, and T+r^ cross section, respectively. The estimates 
of L = 91.5 fb~^ and Ot-t = 0.89 nb are correlated [l^ . 
and the uncertainty on the product Lctt-t is 2.3%. The 
branching fraction upper limits have been calculated in- 
cluding all uncertainties using the technique of Cousins 
and Highland following the implementation of Bar- 
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low The 90% CL upper limits on the r" ^ 
branching fractions, shown in Table ^ are in the range 
(1-3) X 10-7. 
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